DE eng

Search in the Catalogues and Directories

Hits 1 – 5 of 5

1
Self-Supervised Curriculum Learning for Spelling Error Correction ...
BASE
Show details
2
Multi-Head Highly Parallelized LSTM Decoder for Neural Machine Translation ...
BASE
Show details
3
Modeling Task-Aware MIMO Cardinality for Efficient Multilingual Neural Machine Translation ...
BASE
Show details
4
Transformer-based NMT : modeling, training and implementation
Xu, Hongfei. - : Saarländische Universitäts- und Landesbibliothek, 2021
BASE
Show details
5
Probing Word Translations in the Transformer and Trading Decoder for Encoder Layers ...
Abstract: Due to its effectiveness and performance, the Transformer translation model has attracted wide attention, most recently in terms of probing-based approaches. Previous work focuses on using or probing source linguistic features in the encoder. To date, the way word translation evolves in Transformer layers has not yet been investigated. Naively, one might assume that encoder layers capture source information while decoder layers translate. In this work, we show that this is not quite the case: translation already happens progressively in encoder layers and even in the input embeddings. More surprisingly, we find that some of the lower decoder layers do not actually do that much decoding. We show all of this in terms of a probing approach where we project representations of the layer analyzed to the final trained and frozen classifier level of the Transformer decoder to measure word translation accuracy. Our findings motivate and explain a Transformer configuration change: if translation already happens in the ...
Keyword: Computation and Language cs.CL; FOS Computer and information sciences
URL: https://dx.doi.org/10.48550/arxiv.2003.09586
https://arxiv.org/abs/2003.09586
BASE
Hide details

Catalogues
0
0
0
0
0
0
0
Bibliographies
0
0
0
0
0
0
0
0
0
Linked Open Data catalogues
0
Online resources
0
0
0
0
Open access documents
5
0
0
0
0
© 2013 - 2024 Lin|gu|is|tik | Imprint | Privacy Policy | Datenschutzeinstellungen ändern